The Shape of Digits

A Bayesian Topological Data Analytic Approach to Classification of Handwritten Digits

Thomas Reinke

Baylor University

Theophilus A. Bediako

Baylor University

Daniel Lim

Baylor University

August 13, 2025

Contents

  1. MNIST EDA
  2. Tradiotional ML
  3. Proposed Methodology
  4. Analysis
  5. TDA + ML
  6. Results/Future Work
  7. References

Exploratory Data Analysis

Distribution of training labels

Pixel Intensity

Training Data tSNE Visualization

Traditional ML

Neural networks

Feedforward neural network with structure:

  • Input layer: Consists of neurons that receives the input data each neuron in the input layer represents a feature of the input data
  • Hidden layer: One or more hidden layers placed between the input and output layers, responsible for capturing complex patterns
  • Output layer: Final output of the network; Number of neurons represents the number of digits

NN with regularization

  • Depending on model, # network weights > size of training data
    • This leads to overfitting
  • We considered two approaches to overfitting:
    • Dropout learning: Like RF, randomly removes fraction of units in layer during model fitting
    • Regularization: Impose penalties on parameters like lasso, ridge, etc.

Specific NN models considered

  • NN with dropout regularization
  • NN with ridge regularization
  • NN with lasso regularization

Multinomial logistic regression

  • Multinomial logistic regression equivalently represented by NN with no hidden layers
  • Output layer with softmax
    • \(f_m(X) = Pr(Y = m | X) = \frac{e^{Z_m}}{\sum_\limits{l \in K}e^{Z_l}}\)

NN Fitting

  • Train the network for 30 epochs with a default batch size of 32
    • SGD updates weights for each batch
  • Images are presented in batches of 32, and SGD updates weights after each batch
  • Each epoch processes all 60,000 training images
  • Classification correct if largest output value matches target label

Proposed Methodology

TDA Workflow

Analysis

ML Analysis

Need confusion matrices Need accuracy, precision, recall, f_meas

method accuracy
multinomial 0.9856
dropout nn 0.9962
ridge nn 0.9946
lasso no 0.9946

Proposed Method Analysis

Proposed Method Analysis

Proposed Method Analysis

Proposed Method Analysis

metric estimator estimate
accuracy multiclass 0.2023089
precision macro 0.2051790
recall macro 0.2066969
f_meas macro 0.1227207

TDA + ML

Results/Future Work

ML + TDA Results

References

References